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Abstract — We consider the problem of broadcast with common 
messages, and focus on the case that the common message 
rate Ra, i.e., the rate of the message intended for all the 
receivers in the set A, is the same for all the set A of the same 
cardinality. Instead of attempting to characterize the capacity 
region of general broadcast channels, we only consider the 
structure of the capacity region that any broadcast channel 
should bear. The concept of latent capacity region is useful 
in capturing these underlying constraints, and we provide a 
complete characterization of the latent capacity region for the 
symmetric broadcast problem. The converse proof of this tight 
characterization relies on a deterministic broadcast channel 
model. The achievability proof generalizes the familiar rate 
transfer argument to include more involved erasure correction 
coding among messages, thus revealing an inherent connection 
between broadcast with common message and erasure correction 
codes. 

Index Terms — Broadcast channel, common message, individual 
message. 



I. Introduction 

One central theme in multi-user information theory (IT) is 
the pursuit of single-letteiQ characterizations of the capacity 
regions for channel coding problems, or the achievable rate re- 
gions (possibly under certain distortion constraints) for source 
coding problems. However, some useful properties of these 
regions can be identified, e.g., convexity, even when a single- 
letter characterization is not available. An immediate question 
to ask is whether there exist other properties of the capacity 
region that do not rely on a single letter characterization. 

The following question is of interest in this regard: in a 
particular multi-user IT problem, can the achievability of a 
rate vector (iJ*,i2|, ■ ■ ■ ,R%) imply the achievability of any 
rate vector in some regior@ ^(R^R^, ■ ■ ■ , R^), regardless of 
the exact probabilistic channel model? We show that indeed 
this is true for the symmetric broadcast problem, and this 
region can be rather non-trivial. We denote the largest of 
such regions K(R$, -R|, . . . , R* N ) as C(R\, R%, . . . , R* N ) in 
a channel coding problem, and call it the latent capacity 
region implied by (R*, R%, . . . , R^)', the latent achievable 
rate region can be similarly defined, possibly under certain 
distortion constraints, for a source coding problem though it 
is not our main focus. 
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The emphasis on single letter is largely because such kind of characteri- 
zation is usually computable. 

2 Apparently the region defined by Ri < R* is implied in a channel coding 
problem, but this trivial case is not interesting. Note here we do not take the 
subscript of rate R* to have any specific meaning associated with the user 
indices, but merely as an integer label to enumerate the rates in question. 



For broadcast and multiple access channels, a precise prob- 
lem formulation was given in a recent work by Grokop and 
Tse ID, called multicast region, which provides a framework 
to answer the above question. Complete solutions were found 
in JT] for broadcast and multiple access channels with two 
and three users, but the problem remains open for more than 
three users. We believe this problem formulation reveals a 
more general concept not limited to only these two channels, 
and thus rename it as the latent capacity (or latent achievable 
rate) region problem to make explicit this generality. Our 
perspective is different from [1| in that we wish to highlight 
the importance of the latent capacity region concept in its 
"maximum implication" meaning, and thus we shall define the 
region in an alternative (but equivalent) manner to emphasize 
this perspective; our interest in this problem is partially due to 
an observation made during an earlier work [0, as we shall 
discuss shortly. 

One may wonder how a single achievable rate vector 
(R*, . . . , Rpf) can imply the achievability of a certain 
region. In some cases, it is perhaps best explained by the 
familiar rate transfer argument, that the rate to transmit com- 
mon messages can be used to transmit individual messages 
instead, and vice versa. For example, for a two user broadcast 
channel, if a common message rate R^ ± 2 j, and individual 
message rates -R/u and R^ 2 } are achievable, respectively, then 
it is not difficult to see that the region of (R{ 12 },R{i}, R{2}) 
given below is achievable by transferring between common 
and individual rates (see also 0]) 

R{i.2} + R{i} < R*ii,2 



R*r 



R{i} 

#{1,2} + R{2} < #{1,2} + #{2} 
#{1,2} + #{1} + #{2} < #{1,2} + #{1} 1" ll {2}- 

However, for more than two users, such a naive rate transfer 
argument is not sufficient, and additional processing is needed, 
as observed in H] for the three user case. In fact, this was 
exactly the perspective taken in Qj, where the goal is to 
exhaust all such rate transfer operations. The perspective taken 
in U and that taken here are complementary to each other, and 
one may suit certain problems better than the other. Because 
of this relation, it is not surprising that the achievability 
proof of our result also relies on a generalized version of 
rate transfer operations. We shall show that when more users 
are involved, such generalized rate transfer operation requires 
strategic application of erasure correction codes, which reveals 
an inherent connection between erasure correction codes and 
broadcast with common messages. More specifically, in this 
work, we shall largely stay in the framework of [1], and 
provide a complete solution to the K-user broadcast channel 
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Fig. 1. The bold curve gives the rate region: while the left one is possible, the 
right one is impossible for the successive refinement source coding problem. 
The thin lines give the latent capacity region associated with each small black 
dot. 



latent capacity region problem under an additional symmetry 
constraint, whereas only cases with two and three users were 
solved in [ 1 1 without such a constraint. 

The characterization of latent capacity/rate region is impor- 
tant in multi-user IT for two reasons. First, it may facilitate 
finding a single-letter characterization or an approximate char- 
acterization. For example, a rate-distortion region characteriza- 
tion for the problem of multi-stage successive refinement with 
degraded decoder side information was given in the form of 
bounds on sum-rates J2) as 



E 



R l >Yj{X;W m \W 1 ,W 2 ,...,W m - 1 ,Y m ), 



=i 



1 < m < N. 



On the other hand, it seems impossible to establish directly the 
converse for a characterization in the form of bounds on each 
individual incremental rate [3j, despite the fact that the two 
characterizations are equivalent flZ). This is not a coincidence, 
and it is not difficult to show that the latent rate region for 
this problem has exactly the following form, assuming non- 
negativity of the rates 



i=l i=l 



1 < m < N. 



(1) 



Intuitively, when a rate vector (iZJ, . . . , R^) is achievable, 
its latent capacity/rate region gives the largest achievable 
region thus implied, i.e., maximally utilizes it, which may 
help simplify the representation of the region when taking 
the union over auxiliary random variables. Similarly, when an 
approximate characterization is needed, a good inner bound 
may be found by choosing one or several good (auxiliary cod- 
ing) distributions in an information theoretic coding scheme 
which lead to one or several rate vectors, and then taking the 
convex hull of their latent rate regions. One simple example 
is in IIU, where an approximate characterization for the side- 
information scalable source coding problem was given for 
general sources under the squared error distortion measure , and 



the inner bound approximation is exactly the latent capacity 
region implied by a single rate pair. 

The second reason making this concept important is even 
if it does not lead to a single-letter characterization or an 
approximate characterization, it can still provide insights into 
the problem. One such example is that the capacity region 
can always be written as the (possibly uncountable) union of 
latent capacity regions, which places certain constraints on 
the geometry of the achievable region. For the above example 
of successive refinement source coding, we show in Fig. [T] a 
possible rate region on the left, and an impossible rate region 
on the right. The one on the right is impossible because the 
black dot is in the achievable region, thus the latent capacity 
region implied by it (given by the thin line) must be also in 
the region, which is not satisfied by the region depicted on 
the right. This important observation was also discussed in 
(T) (see Corollary 4.3), and we do not elaborate it further. 
Nevertheless, it is rather clear that the latent capacity region 
indeed provides fundamental and useful property of the rate 
region, in addition to the well-known convexity. 



II. Problem Definition and Preliminaries 

We first define the symmetric broadcast problem, and then 
introduce the notion of latent capacity region in this context. 

In a general K-usei broadcast channel, the conditional 
probability distribution is given as 

p(yx[l, 2, . . .], y 2 [l, 2, . . .],..., y K [l, 2,...} \x[l, 2, . . .]) (2) 

where the index in the bracket [1, 2, ...] is used to denote time; 
the random variables have alphabets 3£ , . . . , and the 
receivers are indexed as 1,2, . . . , K. The alphabets can be 
discrete or continuous, and the channel can be memoryless or 
otherwise; for our purpose, it is perhaps beneficial, though not 
necessary, to limit the attention to cases where the channel 
transition process is (block) stationary and ergodic. We use 
script letters to denote sets, and particularly, A and B are 
reserved for subsets of Ik = {1,2,..., K}, i.e., 



A,BC{1,2,...,K}. 



(3) 



|„4| is used to denote the cardinality of set A. A length-n vector 
X[l, 2, n] is sometimes written as X n ; for a K dimensional 
vector (Ri, R2, Rr), we sometimes write it simply as R. 

Let {Wa,A C I k } be 2 K mutually independent and 
uniformly distributed messages, where Wa is the message 
intended for all the receivers in the set A; for notational 
convenience, we include W§ but will assume it to be a 
constant. For each k = 1,2, ... ,K, define the set of random 
variables 



Wfc = {W A ,A: k G A}. 



(4) 



Thus Wfc is the collection of messages that the fc-th receiver 
should decode. We also define the following set of random 
variables 



Wk = {W A ,\A\>k}. 



(5) 
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More specifically for K — 3, we have 

Wi - {Wi.Wu.Wia.Wis,} 

W 2 = {14^2,^12, W 23 ,W 123 } 

W 3 = {W 3 ,W 13 ,W 23 ,W X23 } 
Wi = Wi u w 2 u w 3 
W 2 = {w 1 2,Ty 13 ,w 23! Wi23} 
W 3 - {^123}, 



(6) 



where we have slightly abused the notation by writing, e.g., 
W{i\ as W\. The sets X™ and X i are defined similarly for 
length-n random vectors. In this work, we only consider the 
case that the rates of messages Wa are the same for all such 
messages where the set A has the same cardinality. More 
formally, the problem is defined as follows. 

Definition 1: An (n, R±,R 2 , . . . , Rk ) symmetric broad- 
cast code consists of an encoder 



/ n * 



2 nK \A\ 



3£ r 



ACI K 



where i?g = and K decoders, 



9k ■■ 



n 

-4:fce.A 



(7) 



(8) 



resulting in the decoded messages at the fc-th receiver {Wjt,,4 : 
k G A}, and the decoding error probability of at least one 
message at one receiver 



K 



p(n) 



Pr 



IJ U { W A^W k , A } 
,fe=l A:k£A J 



(9) 



Definition 2: A rate vector R is symmetrically achievable 
if there exists a sequence of (n, R) codes with pj^ — > 0. The 
closure of the set of symmetrically achievable rate vectors is 
called the symmetric broadcast capacity region, denoted as 



C 



p(,yx,V2,—,yK\x)- 



or simply as C p 



Note that secrecy constraint is not considered in the defini- 
tion. Next we define the latent capacity region for this problem. 

Definition 3: For a given rate vector R*, the collection of 
rate vectors 1Z(R*) is called the latent capacity region for 
symmetric broadcast implied by R*, denoted as C(R*), if the 
following two conditions are satisfied (i) For any broadcast 
channel, R* G C p implies 1Z(R*) C C p ; (ii) There exists a set 
of channels {p x }, such that R* G C Px and 1Z(R*) D f] x C Px . 

For the second condition, we essentially wish to find one 
particular channel such that TZ(R*) D C p . However this 
does not quite serve the purpose since this channel might be 
difficult to realize, however it can always be approximated 
by a sequence of channels. The above definition is slightly 
different from the one in [1|, which is 



C(R* 



n c v 



(10) 



p:R'ECp 



It can be easily verified that they are equivalent. The problem 
we wish to solve is the characterization of C(R*). It is clear 
that the region C(R*) is uniquely defined for any R*, and 



thus the problem is meaningful. 

Definition [3] makes clear the "maximal implication" mean- 
ing of the latent capacity region. In multi-user IT, usually 
a coding scheme is given by fixing some auxiliary random 
variables, and then showing a single rate vector is achievable 
with certain random codes; the task of maximizing the impli- 
cation region of this single point is sometimes mingled with 
the conditions under which this single point is achievable. The 
concept of latent capacity region can be used to delineate them. 

The following lemma is needed in the converse proof. 

Lemma 1 (K-way submodularity): Let {Ui,i = 
1,2, ...,N} be a set of mutually independent random 
variables, and {Vi,i = 1,2, ...,N} be a set of random 
variables jointly distributed with it. Let Qi, i = 1,2, ...,K 
be subsets of Xn- Then 



K 

E 

fc=i 



H(Vi,i€gk\Ui,ieg k ] 



K 

>J2 H ( v i> ie Gk\Ui,ieg k ), 
fc=i 



where 



U (S n ng J2 

{h j2,...,jfe}cijf 



(11) 



ng 3k ). (12) 



This lemma is a direct consequence of the sub-modularity 
of the conditional entropy function, when the random variables 
being conditioned on are independent (a proof is given in 
Appendix [A}, and the K-way submodularity property of any 
submodular function given in |0. 



III. Main Result 



Our main result is a complete characterization of the latent 
capacity region for the symmetric broadcast problem. To 
present this region, a few more quantities need to be defined 
first. Let us define the following up-exchange rate for i < j 

-1 

1 i — 1 \ 

(13) 



J %,3 



(14) 



k - i\ vj-r 

J - i) \3 ~i. 
and the down-exchange rate for i > j 

and define 0^ = 1. The up/down exchange rates </>jj 
essentially describe the ratio when converting certain type of 
messages into other types. For example when K = 3, the 
common message Wi 2 s can be used to convey individual in- 
formation to the three users, and vice versa, but the conversion 
of such rates is not always ratio one. It will become clear in 
the achievable proof how such conversion can be done in a 
most efficient manner. 

Define C*(R*) to be the set of rate vectors R satisfying the 
following conditions with some K 2 non-negative quantities 
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K 

R*>Y, r ^' i = l,2,...,K, (15) 

K 

• II, • X]o,.,r, ; . j = 1,2,..., if. (16) 

i=l 

Roughly speaking, the rate r,.j is that taken from level-i 
rate R* but used to transmit level- j messages. We have the 
following theorem. 

Theorem 1: For any non-negative rate vectors 
(Rl,R$,...,R K ), we have 

L{K 1 ,H a , . . . ,K K ) — L {K X ,H2,...,H K ). (1/) 

Example: for K = 2, it is straightforward to see: </>i.2 = 1, 
i.e., the same amount of individual message rate for each user 
can be used to transmit a common message; and $2. 1 = 1/2, 
i.e., to split a common message into two equal parts, each to 
transmit a separate individual message for one user. 

Example: for K = 3, it can be verified using Fourier- 
Motzkin elimination [7] that C*(Rl,R*, . . . , R* K ) is given by 
the non-negative rates satisfying 

3i?i + 6.R2 + 2i? 3 < 3i?t + &R* 2 + 2i?3, 
2i?i + 2i? 2 + 1R 3 < 2R\ + 2R* 2 + 1R^, 
lRi + 2R 2 + 1R 3 < 1R1 + 2R* + 1R*, 
3i?i + 3i? 2 + li? 3 < SRI + SR* 2 + IR* 3 - (18) 

A typical shape is given in Fig. |2] with (i?^, R%, R^) = 
(1,2,2). The computation is tedious and thus omitted here. 
The same result can also be reduced from that given in JT] for 
the asymmetric case. It is clear that this region is non-trivial, 
and it is not at all clear a priori why these rate combinations 
should be considered. 

In (T), the region is characterized by investigating the 
distinct universal encoding/decoding operations, which leads 
to the concept of extremal rays. Because the latent capacity 
region in question is a polytope, it can be characterized by 
its faces, edges, or vertices. The extremal rays are essentially 
the edges of this polytope. However this proof approach in 
0] appears rather difficult to generalize for more than three 
users since the number of edges quickly becomes very large, 
and thus we introduce the parametric characterization (TT31 and 
( [ToT l to avoid this difficulty. 

Notice that the exchange rate is pairwise, suggesting in this 
symmetric setting there is no need to convert rates jointly, e.g., 
use W12 and W3 to send the same message W123. In the rest 
of the paper, we shall prove Theorem [1] The naive approach of 
finding the planes of the rate region and derive its upper and 
lower bounds is not appropriate for general K, particularly 
for the purpose of converse. Instead, we utilize the structure 
of the region C*(R*) to give a proof. 

IV. Proof of the Forward Part For TheoremQ] 

The proof of the forward part of Theorem[T] i.e., the fact that 
C*(R*) satisfies the first condition in Definition [3] is relatively 
straightforward. 




Fig. 2. The latent capacity region implied by rate vector (1, 2, 2). 



Proof of forward part for Theorem [7} Since (R*) is 
achievable on any channel, there exists a sequence of codes 
with such rates with Pi™^ — > 0, and we will use these codes 
to construct a set of codes to approach any rate vectors in 
C*(R*). This is done by essentially relabeling and adding 
erasure correction codes on the messages. 

Observe that the messages |^4| = i) can also be used 

to transmit common messages to the subsets with cardinality 
smaller or larger than i. Moreover, we can use part of the 
rate R*, denoted as r,j, for this purpose, to transmit some 
messages {W^, \A\ = j}, thus increasing Rj. Such an 
operation will cause a conversion of rate for i-user subset 
messages into rate 4>ijrij for j-user subset messages, with an 
exchange rate <pij. The region C*(R*) is precisely the result of 
allowing this kind of pairwise exchange on the rate vector R*. 
Thus we only need to show that the exchange rates <fri j given 
before Theorem Q] is indeed valid, then the existing sequence 
of channel codes can be used directly. 

It is clear that we only need to consider the following 
problem: on a channel with Ri = R and Rk = for fc ^ i, 
how do we transmit messages {Wa, \A\ — j}, and how much 
rate Rj can be supported? We will only need to distinguish 
two cases i < j or i > j, since it is clear 4>i,i = 1. 

We first consider the case i < j. For a subset B of Ik where 
\B\ = j, there are a total of (^) subset of B with cardinality i; 
denote the collection of such subsets as 2 B ' % . For a particular 
user k £ B, it can decode (with high probability) the messages 
{Wa '■ k G A C £>}, i.e., ( 3 Zi) such messages. To transmit 
the common message if we can guarantee that when 
receiving any ({z]) messages out of the messages in the 
set 2 B '\ the message is decodable, then it is clear that indeed 
any receivers in the set B can decode the message Wb- This is 
an erasure correction problem and a m), QZi)J maximum 
distance separable (MDS) code can satisfy this requirement, 
which indeed exists when the codeword length is sufficiently 
large. Furthermore, since each subset A of cardinality i is a 
subset of ( ■ sets °f cardinality j, only (^T/) of the rate 
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Rj( can be used for each MDS code. This yields 

Next consider the case i > j. Let B be a subset of Ik where 
\B\ = i. The common message B can be shared uniformly 
between its Q) subsets of cardinality j, for transmitting their 
"individual" message. Since each subset A of cardinality j is 
a subset of distinct sets of cardinality i, it can take part 

in such sharing (^JT/) times. This yields 

Rj = Q (f-/)^ = (20) 

Taking into account the existence of good MDS code, and the 
fact that C p is a closed set, the proof is complete. ■ 
In [l], it was observed that in order to efficiently transfer 
rates, sometimes a modulo two addition is needed, similar 
to that seen in butterfly network of network coding [6|. The 
MDS codes we use in the above proof can be understood as 
a generalization of the modulo two addition, which itself is 
essentially a (3, 2) MDS code. It is worth noting that other 
coding/processing may also be useful for converting rates, 
however, MDS codes are sufficient in solving the symmetric 
broadcast problem. 




Fig. 3. The deterministic broadcast channel K = 3 from [T]. 

This is indeed our proof approach, however before giving 
the rather long proof for the general case, we first prove a 
few rate combinations for K = 3, which illustrates the basic 
techniques as well as facilitates better understanding. Though 
the proof of the case for K = 3 can also be found in HI, 
our proof given here is different and in fact more structured, 
which is geared toward the general case. After this example, 
a few necessary tools and intermediate results are provided, 
and finally we give the converse proof of Theorem Q] 



V. Proof of the Converse Part For TheoremQ] 

The converse proof of Theorem Q] requires more work. For 
simplicity we shall assume 2^'s are all integers; if this is not 
the case, a sequence of channels need to be considered, and 
we shall return to this technical point after the proof. 

We only need to provide one particular channel that R* G 
C p and 1Z(R*) D C p . The channel is the deterministic one 
considered in (T), extended to the if -user case; see Fig. [3] 
for the case K = 3. More precisely, let the channel input be 
the collection of {X_a,A C Ik}- The alphabet of Xj( where 
\A\ = k is I 2 R' k - The fc-th channel output Y k is given by 

Y k = {X A : k G A}. (21) 

Denote this deterministic channel as p* . In order to prove the 
converse part for Theorem Q] we need to establish C*(R*) 3 
C p * for this channel. 

For any A = Ax, A 2 , . . . , Ak where Ai > 0, define the 
following quantity 



and similarly 



A" 

B C .(A)= max VA fe i? fe , 
v ' fc=i 



K 

B C {A) = max V A k R k . 

p k=l 



(22) 



(23) 



It is clear that both C*(R*) and C p * are convex regions, and 
thus if we can prove the following theorem, then the converse 
of Theorem [T] directly follows. 

Theorem 2: For any A where Ai > 0, 



A. Bounding Two Rate Combinations for K = 3 

We give an outline of the proof for the first two inequalities 
in the example given after Theorem Q] 
Proof: 



3nR{ + 6nR* 2 + 2nR* 3 

3 -, 3 

> 

3 

(a) 2 
> - 



i=l 
3 



i=l 



f \Y,H{X?\X^Wx2*)-n5 



= 2nRx + AnR 2 + 2nR 3 + t ^ h(X? \Wi) 



i=l 



1 3 

- £ H(X?\Wi2 3 ) - H(X? 23 \W 123 ) - nS 



> 2nRx + AnR 2 + 2nR 3 + \h (X^ 23 \W 123 ) 
1 3 

+ -Y} H (W l \W l23 ) + H{X?\W i )} 

-H(X? 23 \W 123 )-n6' 
= 3nR 1 + 6nR 2 + 2nR 3 - nS' 

"l 3 1 

-j2H(xrm)--H(x? 23 \w 12S ) 



3 

L i=l 



B C *{A)>B C {A). 



(24) 



(d) 

> 3nRx + 6nR 2 + 2nR 3 - nS' , 



(25) 



where (a) is by Fano's inequality, (b) is by adding and 
subtracting the same term, (c) is by applying Fano's inequality 
on the third term, and noticing that Lemma Q] together with 
the fact of the channel being discrete implies that, 



1 2 3 4 5 



K-1 K 



3 

E 

i=l 



> H(XZ\W X ) + H(X n l2 \W 2 ) + H(X? 23 

> m a x{H(X[ l 23 \W 123 ),H(X^\W 2 )} 



W 123 ) 



(26) 



and (d) is again by the inequalities in (|26l l. This completes 
the proof for the first rate combination. For the second rate 
combination, we have 



6nRl + 6nR 2 + 3nR^ 

3 



i.=i 

3 



3 

E 

i=l 



H(X?) 



> H ( X l l \ X 2W2) + 3nRi + 6nR 2 + 3nR 3 

i=l 

+ H{X 2 \W 2 )-nS 

3 

= Y,H{X?Xl\W 2 ) - 3H(X 2 \W 2 ) 

i=i 

+ ZnRx + 6nR 2 + 3nR 3 + H(A%\W 2 ) - nS 

> QnR\ + 6nR 2 + 3nR 3 — nS 



3 

E 

.»=! 



H(X?X 2 |W<W a ) - 2H(X 2 \W 2 



(6) 

> 6ni?i 



6nR 2 + 3ni? 3 - nS' , (27) 
where (a) is because of (l26*l l. and in (b) we applied Lemma Q] 



3 

E 

i=l 



H{X?X 2 \WiW a ) > H(X 1 \Wi) + 2H(X 2 |W 2 ), (28) 



and then omit the first term since the channel is discrete; the 
rest of the inequalities in ( f27l > are by Fano's inequality. ■ 
This proof illustrates several main components of the proof 
for the general case. Firstly, the rate combination needs to be 
written as summations under appropriate proportions, secondly 
the K-way submodularity lemma needs to strategically used, 
and thirdly there are connections between different layers of 
messages and thus terms may be canceled among them. For the 
general if -user problem, the bounding becomes much more 
complicated, and we will rely on the optimal solution Be* (A) 
to provide necessary structure and guidance. 

B. Several Properties of faj 

We begin with a few properties on the exchange rate (f>ij. 

Lemma 2: For any integers i,j,k such that 1 < i < j < 
k < X, we have (f>i,j4>j,k = <f>i,k- 

Lemma 3: For any integers i,j,k such that 1 < i < j < 
k < X, we have (pk,j<pj,i = §h,i- 

Lemma 4: For any integers i,j such that 1 < i < j < K, 
we have (f'ij'Pj^ = < 1- 



Fig. 4. An illustration of the optimal extremal solution structure. The longer 
and bolder marks give the set S. 



Lemma 5: For any integers i, j, fc, we have 



k,j<pj,k, 



with equality only when the sequence k) is monotonic. 

Lemma 6: For any k > j, we have (fc + 1) ( fcZi ) <fe+i,j = 

Lemma 7: For any i < j, i^-i) = 

The above lemmas (particularly Lemma |2]|5]l may be 
best understood as a currency exchange system where up- 
converting (or down-converting) many times results in the 
same final exchange rate as a single step conversion, but 
up-converting mixed with down-converting to the original 
currency results in a loss. The proofs of these lemmas are 
given in Appendix [B] 

C. Extremal Solutions and the Effective Rate Set 

To prove the converse part of Theorem |2] we proceed in 
two steps: first we identify some special optimal solutions for 
the maximization problem (|22| > with certain desired properties, 
then show that Be* (A) is an upper bound to the quantity 
Bc(A). In this subsection we discuss the first step. 

Definition 4: A non-negative setting of satisfying dT~5b 
is called extremal if the following conditions hold (i) For each 
i = 1,2, . . . , K, there exists a unique j 6 Ik such that r,,j = 
R* and r ijfe = for k ^ j. (ii) If nj = R* > 0, then 
Tjj = R*. (iii) If nj = R* > 0, then for any k such that 
max(i,j) > k > min(i,j), rkj = R^- 

Lemma 8: The solutions to the maximization problem (l22l 
include one that is extremal. 

The lemma is intuitively true since a linear optimization 
problem has an optimal solution at its corner point. The 
concept of extremal solution makes the definition of corner 
point in the problem context more precise. A proof is given 
in Appendix iBl 

Definition 5: In an optimal extremal solution, the effective 



rate set is defined as £ = {i E Ik 



> for some j}. 



The elements of £ in an increasing order are denoted as 
e\,e 2 , . . . , e\ £ \. 

Lemma [8] implies there exists a specific structure of rate 
exchange in the optimal extremal solutions. 
Lemma 9: For an optimal extremal solution: 
> There exist a partition of the sequence 1, 2, . . . , K, la- 
beled as 5i,c>2, ■ ■ ■ , <Si£i, each consisting a consecutive 
sequence of integers, and £ Si. 
• For k e Si, we have rfc ei = 

This structure is analogous to scalar quantization to some 
extent, as illustrated in Fig. H] 

D. Proof of the Converse Part of Theorem \2\ 

Proof: For a fixed vector A, let {fij} be an optimal 
extremal solution for the maximization problem (l22t . and let £ 
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be its effective rate set and let <Si, 1S2, • • • j Si£\ be the partition Lemma [TJ on the second term. For notational simplicity, we 
sets; for convenience, denote the smallest element in the set shall ignore the small quantity 6 in the sequel. 



Si as li and the largest element as Ui. Assuming a sequence Slightly further expanding the first term in (132) and sub- 

of length-n codes is given with diminishing error probability, stituting it in L k give us d33l . More generally, we claim that 

Let Xi and Xi be defined similarly as Wj and Wj. The proof for m such that u k — 1 > m > e k — 1, ( [34-b holds, which we 

consists of two layers of inductions. We start from the inner prove by induction. Clearly it holds for m = u k — 1 since it 

layer, and then put the pieces together in the outer layer. is exactly ( T33l > in this case. Suppose it holds for m = m*, we 

Define the following quantity for k = 1,2,..., \El for ^ P rove h ^ holds for m = m* - 1. Putting (El into 

which lower bounds will be derived g' we have «3 given on the next page. In order to simplify 

(135), first notice that a fc , m » + b k , m *+i = b k , m *, and 



K 



+ a Mfc ^ff(^|< fc+1 W Ufc+1 ), (29) (a)^^_ A efc+1 ^ fc , efc+1 ^ m ., Ufc 

i=l ^e fe 

where W 1 J 4-e/i+i ( Aro* 1 ei s +i 



— Vm* ,e k a 



(36) 



a fcj = |K-h " ^if-n > 3 = e k ,-..,u k - where (a) . s by Lemma □ and (b) is by Lemma [2] It follows 

, that 



and for convenience we have defined A e .„... — and 



a k , m . ]T ^(Wi|W ro .+i) + 6 fe , m *+i £ (W 4 |W m . +1 ) 

i=l i=l 



«),■•...,, = u. iNoie mat an me coerncienis in rroni or me , v-^ ,71,., ittt \ ly ~ 1 \ 

1 1 l+ f t - t - tU =6fe,™*> H (Wi W TO » ) + nKb k . m ,[ )Rm*, 

entropy functions are non-negative: those in the first sum- ' ' \m* — 1/ 

mation are straightforward to verify by using the definition „ 

of (f>i.j, and for the last term we only need to observe that = b k , H(W \W 

A ek (j)u k ,e k > A ek+1 (p Uk ^ k+1 by the optimality of the extremal '™ 4^ 

solution. For convenience let us also define / 



7 A Y3,ek s±e k+1 (Pu k ,e k+1 ST-^ . \ A, 

Ae k\u k -i) i—j Furthermore, notice that 

(31) m * _ _ 

which are clearly non-negative quantities. We are interested in °fc,m* 2_j ^(^i 1^*) — a,k, m * m H(X m * +1 \W m *+i 
these quantities L k 's because they are directly related with the 
rate combination being considered, as we shall see shortly. 



nK ( <f> m , .„ - -- et+i ; m - ek+1 ] R m > . (37) 



i=l 



We start by writing the following 

K ™* 



i=l 

if 



+ (& fe *)#(«*m*+l|Wm*+i} 



-^f(a-; +1 |w j+1 ) 



Y^H(xrx; +1 m +1 )-KH(x; +1 \w J+1 ) ~X h JE^T H{Xm ' +llWm * +l) ' (38) 

3re the last step is d 

bk, m *+i — a, km *m 



Uk—lJ 

^ k k where the last step is due to 

i=l i=l 



(6) 



A' j (.TO*/ ^ e fcVu fc -lJ lm*-lj 



> £ff(Wi|w i+1 ) + £ir(*rivi>i) = _£ 



e fc+ i Yu k ,e k +i 



jff(F; +1 |W i+ i)-n« J (32) 



^ e fc(« fc -i) 

j+i 1 ' K y u ' w / where the last equality is by Lemma |6l Combining d35]l, (l3~7li 

where (a) is by Fano's inequality, and (b) is by applying and ( 1381) . we have ( l40b , proving that the claim d34l l is indeed 
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true. 



following inequality holds 



Letting rn = e k — 1, we can write fiTb on this page. By 
breaking the second term as given in (l42t . where in the last 
step we apply Lemma [7] and Lemma [2] and noticing that for 
the third term 

b k , £k jr h{X\w 1 ) > b k , ek e H(x™m), 

i=l i=l 

implied by the discrete nature of the channel, we can further 
write (l43l l on the next page. 

This concludes the inner layer induction, and next we turn 
to the outer layer. First notice that the optimality of extremal 
solution and Lemma [9] together imply that 



\£\ 

i=i jes. 



(44) 



We first write ( |43T l where the inequality can be justified 
as follows. Observe that in the second summation, for any 
k > eigi, the random variables Xj, with \A\ = k appear 
only in the last K — k + 1 terms in the outer summation. 
Each inner summation has a total of K(^~^) such terms, 
which implies such random variables are counted a total of 
j4 e|e| K(f)j >e , £ , times. Thus by the cardinality of the alphabets, 
the normalized entropy is upper bounded by Through a 
similar argument, it is not difficult to verify that for k < eigi, 
all the terms are accounted for. Furthermore, notice that by 
the optimality of the extremal solution, for any j G Si, we 
have A ei 4>j :ei > A e . £ . (j>j, e , £ , , and thus the first summation is 
non-negative. 

We next apply d43l > with k = \£\ in d45l >. and write (|46| | 
on the next page, because we have u\g\ — K, A e{£l+1 = by 
definition, and 



1 



>|£|.e|£| 



(K-l) (K-l) 
\e\ e \—U \e\e\-V 



(47) 



More generally, we claim for k = 0, 1, . . . , \£ \ — 1, the 



Be (A) > E E [Ae^ - A ek+1 ^ £k+1 ] R* 



»=i jeSi 



\e\ 



i=k+l j=h 

\e k + 1 -l) i=l 



\ek+i — V i=l 



(48) 



We again take an induction approach to prove this claim. 
The claim is clearly true for k = \£ \ — 1. Now suppose (|48| ) 
is true for k = k*, and we seek to show it is also true for 
k = k* — 1. For notational simplicity, let us define 



Cj',fc — A ei (j)j tei A ek + 1 (j)j tek+1 . 

We first prove the following inequality 



(49) 



k" 



k*-l 



A, 



E E c ^* r j * E E :'<; + -^t L **- (50) 



To do this, we need to count in the second term the number 
of appearance of random variables X_a for all \A\ = m, for 
all fixed m, such that m G l Uk , ■ This is similar to d45b . but 
slightly more involved. For m such that u k * > m > e k *, 
it is easily seen that there are a total of bh*,m,K( ~J) such 
random variables in L k », implying the following amount of 
R* is accounted for 



K \m — 1 



.4 



e fc » Ym,e k * 



A e 



1 e t « + iVm,e t « +1 ! 

(51) 

where we have used d36l l. This indeed is the difference 
between the left hand side of (T50b and the first term on the 
right hand side, in terms of R* m . For the case m < e k *, the 
following amount of R^ is accounted for 

A e ,, (K - 1 

K V 771 — 1 



Ae k * 4>m,e k 



™-e k , +1 <¥m,e k * +l , 
(52) 

where we have used the derivation in (|42V This is again 



.4 



Ru 



u k -l K , 

L k >Y, ^j^Hm^ +1 W j+ i) + nK U Uk , ek 

j=e k i=l 

K u k 

+ a ktUk J2H(Wi\W Uk ) + a k)Uk J2H(^\Wi) - u k a k<Uk H(xl k+ i\W Uk+l ). 



e k+1 Vu k ,e k+1 

~aZ 



(33) 



i=l 



m K Uk / A (t> \ K 

L k > E OfcJ E H{Xr\X n j+1 W j+ i) +nK E Ui.eu ~ Rj + b k , m+ i E H(Wi\W m+ i) 

j—Ck i—1 j— m+1 k i—l 

m+1 a ± u k — l 

+ b k , m+ i E #(*riW.) - Uka k , Uk H(X n Uk+ i\Wu k+ i) - ^(K- 1 ) 1 E ff(^+i|W m ). (34) 

i=l e fc\u fc — 1/ i— m+1 
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m — 1 K Km 

L k > E ak >i E H ( x ? i^+i w j+!) + afe - m * 12 ff (Wiiw m .+i) + o fc , m . E ff(*riWi) 

j=ek i—1 i—1 i—1 



uk / /l _i \ ^ 



I in -j ■ J. ' e fc / ... | 

^ / tifc— 1 

6 fc , ro .+i E H (X l \m) - - ^Jk-IT E H(X l +1 \m +1 ). (35) 

i—1 e k\u k — II i—m*+l 



m *- 1 K Uk / A 6 \ K 

L k >Y a ^ 12 H(K\^+iW 3+1 ) + nKY ( " efc+ ^' efc+1 j ^ + E # CWm*) 

j—e^ i—1 j—m* ^ e k / i—1 

rn* , Uk — 1 

+ & kl m. E ff(sriwo - Ufe a fe , Ufc i/(^;; fc+1 |w Ufc+1 ) - ^ +l y: e * +l E #(*r+iiw m ), (40) 

i—1 e k\Ufc— 1/ i—m* 



6fe 



t=l 

Uk — 1 



u fc a fclUfc H(^ fc+1 |W ttfc+1 ) - E ff^ + i|W i+1 ). (41) 

e fcVit fc -l/ i=e fc 



b k , ek J2H{Wi\W ek ) = b k ,e k nK E ( . _! K +6 fe , efc E ff (^l W ' 



3=i* 

J- 



efe— 1 / 




ni^E 








, (e h -l) 










j=lk 





Uj+fefc.e.E^I^)' (42) 



i=l 



precisely the difference between the left hand side of (f50b and in (a) we used Lemma [JJ and (b) is because 

the first term on the right hand side, in terms of i?^. Thus ^ j± 

(EB is indeed true. ~^ b e k *,e k , + , "k + -i \ 

71 \e k m +1 -l) 



nK&Z\) nK{«;\) nK{ e ^_ x ) 

A e k * A e k , +1 ( U fc »-l) 



Now we proceed with the proof of (l48l l through induction = k( k - 1 \ k( k ~ 1 ) \ rx **' e **+i / k-i \ / 

by assuming it holds for k — k*, and write (T53T > on the top of vfc*-i/ \u fc *-i/ \ \e fe * +1 -i/ / 

this page by applying d43l . In order to simplify (l53l l. similar _ ^-e fc * 
terms need to be combined, for which we write (l54l . where ni"f(^~^) ' 



Uk ( A <b \ K lk 

L k >nK h,e k - £k+1 ^ ek+1 Rj + b k , ek J2 H(Wi\W lk ) + b Kek E 

j=l k ^ e fc / i=1 i=1 

- u k a k , Uk H(X k+1 \W Uk+1 ) - ^ +i y: e * +1 E ^(<+i|W l+ i). (43) 

e fc V«fc — 1/ i=e fc 
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\S\-1 

B C *(A) >J2T, [ A ^i- A ^,e m ] R* 



A 



K 



'■If 



<Aj+l, 



K 



1=1 jes. 
\e\-i 

E E 

1=1 jes, 



nK 



E l^-^fJE^i^^) 



]= e \e\ WJ- 



A 



(45) 



|£|-1 U|£| 



-4„ 



^(A)>^ e^^-^i^w+^i e u^- ei£i+ r 3 ' ei£i+i ) A 



, i=l t=l 
4 4 "lei" 1 



x: ^(^T + iiw, + i) 



\U\e\-X) «=e|£ 

K ^ K 

= E T,[ A ^- A ^,e m ]Rj+ A e lel E ^'^ + ^#M^ ff(W,|W,|f|) 
i=l jeS; Ve| £ |— 1/ <=1 

+ ^fer T E^(^|W J ), (46) 

\e| £ |— 1/ i=l 



fc*— 1 "fc* / 4 

i=i jes, j=i fc , v 6fc * 



n \i=l i=l / n 



l) 

Vw fc * -1/ 



^. +1 v^_hL £ ff(^ +1 |W m ) + £ .1. E'V 



ve fc » + i-l/' i=l \e k » + 1 -U i=l 



where the last step is again by Lemma [7] 



Next consider the summation ( 1561 1 where we have split the 
last term and combined it with the other terms, and used (l55t : 
moreover, some terms H(X i \Wi) for i = Ik* + 1, • • • , e^* are 
ignored because they are non-negative by the discrete nature 
of the channel. Observe that for the last term in the right hand 
side of i 



A 



e k * + i 



A 



nK( K -\ 
W*+i-i 



A 



A 



e k r +l Yu k * ,e fc » + i 



K-l 



0. 



(57) 



For the second term in the right hand side of (156) , notice 
that + 1 G <Sfc*+i, thus by the optimality of the extremal 



solution, we have 



.4 



e k * + 1 <Pu k ,+l,e k » +1 > A ek ,cp Uk , +ltek 



(58) 



and thus ( |59l > follows, where (a) is by Lemma [4] and the final 
inequality is by (l58i Now combining d53l l. (l54l i. (|56] >. ( T57] > and 
(l59l l completes the induction proof of d48b for k = k* — 1. 

Writing (l48l l for fc = 0, we have 

\£\ ui 

B C ^A)>Y^ A ^Y.^^ R J 

1=1 j=li 



^E^E^>*^' 

<=1 j=h 



(60) 
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Uk " / A rh- \ A K \S\ u t 

Ae k , E (4W - ) Rj + ^^.e fc . E g OVi|W, t .) + E E'V 



£ (- 4 ".>i..«. -■ 4 ..-«fe..-«)% + ^»t-,«,.i;ff(W.|M'v)+ £ -4,, !>.,.„/!( 



A' 



i = fc*+l j = ( i 



\£\ 



3=h 



nK 



i=fe*+i j=i 



Ve fc . + i-l/ i=l j=2 fc * Ve fc , + 1 -17 w 7 

|f I «i / A a \ K 

( - } E ^, E + 4a-''- ' • + ] E ^ oviiw... ) 

i=fe* j'=(j \ \e fc » +1 — 1/ / i=l 



(54) 



i=i 



VUfc* — 1/ i=e fc * \efc*_|_i — 1/ i=l 



Vtl fc * 



nAT 



^-e fc » +1 _ A ek „ +l (f) Uk , ,e fc » +1 

ll" ^(* _ \) / 

,e k * +1 -l/ \u k *—V / i=e fc 



K-l 



(56) 



A, 



nK( K -\) nK 



^k* ®k* ,u k * — 



Ae k * Uk* ( 4>u k * ,e 



A 



e k * + 1 Yu k * ,e fc » + i 



nK 



IK-1\ A I K—l \ 

W«-i' e fc* 



n ^(e fc , + 1 -l) V 1 u j -11 : • «, 



nKl 



K-l 



(a) ^e fc . + i0ti fc .+l,e fc » +1 ^e fe * u k* 4>u k , ,e k * 4>u k * +l,u k * 
nK {u~-^u k *+l.u k , 



> o, 



(59) 



where the second inequality is because the first term in 
the parenthesis degenerates to zero, and the second is non- 
negative. Notice that for any j G Si, by the optimality of 
the given extremal solution, Aj < A ei cpj^ ei , thus by the non- 
negativeness of rate Ri's, we arrive at 

\£\ Ui K 

b c *{a) >e^E > E A i R i- (6 v 

i—1 j=l% i—1 

This completes the proof. ■ 

For the case that 2- R «*'s are not integers, we can instead 
consider a sequence of channels with memory, for which the 
alphabet sizes are 2 nHi , however, for each n channel use, 



the channel erases (n — 1) of them. This channel is not 
a memoryless channel anymore, however, our definition is 
sufficiently general to include such a case, and the converse 
proof can be used without any change. 

VI. Conclusion 

We consider the latent capacity region of the symmetric 
broadcast problem, which gives the maximum implication 
region for a specific achievable rate vector. A complete 
characterization is provided, for which the converse proof 
relies on a deterministic channel model, and deriving upper 
bounds for any bounding plane of the rate region. The forward 
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proof reveals an inherent connection between broadcast with 
common messages and erasure correction codes. 

We believe the latent capacity region (or latent rate region) 
is a general concept, and can be applied to other problem. 
In |JJ, the multiple access channel is also considered for two 
and three-user case. It is conceivable that the technique used 
in this work can be used to generalize their results for the 
multiple access channel. Another interesting case may be the 
interference channel, where the well-known Han-Kobayashi 
region J9) is indeed the projection of a rate region for the 
coding problem with common messages. A careful analysis of 
the latent capacity region for the general interference channel 
may yield further insight into the problem. 



Appendix B 
Proof of the Lemmas 



Proof of Lemma [2} 

'K-i\- x {j-l\ /K-jyWk-V 

J — * / \j — v \k — 3 ) \k — j / 
(j - i)\{K - jW - 1)1 (fc - j)l(K - fc)!(fc - 1)! 
(K - - i)!(i - 1)! (K - j)\(k - i)!(j - 1)! 
(K - k)\(k - 1)\ (k - i)\ 



(K-i)\(i-l)\ (k-i)\ 



(66) 
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Appendix A 

SUB MODULARITY PROPERTY OF CONDITIONAL ENTROPY 

Lemma 10: Let Ui, U 2 , . ■ ■ , Un be a set of mutually in- 
dependent random variables, and let Vi , V 2 , ■ ■ ■ , Vn be N 
random variables jointly distributed with them. Let Q be a 
subset of Zjv> i- e -> G Q %N- The conditional entropy function 

Hy\u(G) — H(Vi, i £ G\Ui, i £ Q) is a submodular function, 
i.e., for any Q\,Qi C X N , 

h V \u{Qi) + H V \u{g 2 ) > H vlu {g 1 u g 2 ) + H V \ U {g 1 n g 2 ). 

Proof: Notice that 

h v \u{Qi) + H vlu (g 2 ) 

= H(Vi,Ui,i g Gi) + H{Vi,Ui,i G g 2 ) 
-H(U i ,i&g l )-H(U i ,i&g 2 ), (62) 

and 

Hv^igiug^ + Hy^&ng^ 
= H(Vi, u l7 iGGxug 2 ) + H(Vi, Ui, i£ gin g 2 ) 
- H(Ui,i g g x u g 2 ) - h{u %1 jeftn g 2 ). (63) 

The mutual independence among Ui's gives 

H(U i ,ieg 1 ) + H(U i ,ieg 2 ) 
= H(Ui, teg!ug 2 ) + H(U it i e ft n Q 2 ). (64) 

The submodularity of unconditioned entropy function of ran- 
dom variables is well-known (8), which gives 

H(v i ,u i ,i€g 1 ) + H(v i ,u i ,i€g 2 ) 

> H(Vi, Ui, i g g% u g 2 ) -\- H(Vi, u u < g g x n g 2 ) (65) 

and the proof is thus complete. ■ 



Proof of Lemma \3} 



k \ ( K -J)( 3 
k — j) \k — j ) \j — i 
(k - j)\j\(K - j)\ (j-i)W.(K-i)] 



K 
j - 



k\(k - j)\{K - k)\ j\(j-i)\{K- 
i\(K-i)\ (k-i)\ 



J) 



k\{K-k)\ [k-i)\ 



(67) 



Proof of Lemma® By the definition of tfiij, it is easy 
to verify that 



< 1. 



(68) 



Proof of Lemma [3} The case i = k is exactly Lemma 0] 
thus we only need to consider the case i ^ k; we may also 
assume j ^ i and j ^ k since these cases are trivial. The order 
of i,j,k can be arbitrary, but since the proof only relies on 
Lemma [2] [3] and |4] we may assume without loss of generality 
i < j. Thus we have the only three cases. (l)fc<i<j:by 
Lemma [3] and |U we have <f> ijk > <t>i t h<t>i,3<t>j,i = i>i,j^>j,k- (2) 
i < k < j: by Lemma |2] and [4] we have (fn^ > <fii } k<t>k,j4 l j,k = 
(t>i,j<pj,k' (3) i < j < k: the equality is implied by Lemma [2] 



(fc+1) 



Proof of Lemma [6} 

(k + l)(K-l)\(K-j)\(k + l-j)\j\ 
(fc - 1)\(K - fc)!(fc + 1 - j)l(K - fc - l)!(fc + 1)! 

(k - iy.(K - ' 



(k - 1)\{K - k)\{K - k - l)lkl' 
Similarly, we have 



(69) 



proving the lemma. 



k(K - iy.(K - mk - 

k\{K-k- l)!(fc- j)\{K-k)\k\ 

(K - 1)\(K - 
(k - l)l(K - k)l(K - k - l)\kV 



(70) 
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Proof of Lemma [Tf We only need to write the following 

k-i\ (K-iy 1 _ (k - iy.(j - i)\(k - j)\ 

(i - 1)\{K - i)\(K - 1)! 

{K-j)\{j-i)\ 
(K-i)\ 



Proof of Lemma® Suppose an arbitrary optimal solution 
of the maximization problem (l22l is given, we shall next 
transform it into an extremal one which is also optimal. 

For condition (i), we may assume R* > because otherwise 
the statement is trivial. Observe that for any optimal solution, 
the second inequality in (fTSI l must hold with equality, because 
otherwise the quantity being maximized can strictly increase. 
First suppose for certain i, there exist distinct j% , ]2 such that 



j 1 > and rij 2 > 0, then we must have 



= A 



(72) 



because otherwise, e.g., if < held, then letting r\ ■ = r i j 1 + 
ri j 2 and r[ - t = strictly increases the quantity being 
maximized in (l22l . However, if d72l is true, the new solution 
given above does not decrease the quantity being maximized, 
thus a new solution can be found such that there exist no such 
two distinct ji,j2- Given this is true, it is clear that for each 
i, letting the unique j for which n t j > be R* is an optimal 
choice. Thus condition (i) is satisfied by some optimal solution, 
and from here on, we shall only consider such solutions. 

For condition (ii), we may assume R* > since otherwise 
the statement is trivial. Suppose condition (ii) is not true, i.e., 
for some k ^ j, rj <k = R*, then the optimality of the solution 
implies 

Aj < A k <p jik , (73) 
Now we claim that the new solution with r' ,, = R* and 



1,3 



(with other fjj values unchanged) can not decrease 
the quantity being maximized. To see this, we only need to 
observe that 

A k (f>i,kR*i > A k <t>i,j<t>j,kRt > Aj&jR*, (74) 

which is by Lemma [5] and ( |73l . Thus the conditions (i) and 
(ii) are indeed satisfied by some optimal solution, and from 
here on we shall only consider such solutions. 

For condition (iii), we only discuss the case i < j, because 
the other case i > j is similar. The fact that r.j j > 
implies AiR* < Ajtfii jR*. We may assume i?£ > because 
otherwise the statement is trivial. Take an arbitrary k, such 
that i < k < j, we may have r k ji = R^ for some j', and the 
value of j' may be j' < i, i < j' < k or j' < k; note that we 
can assume j' ^ i since condition (i) afore-proved. It is easy 
to see that we must have Aj > and Aj> > 0. The fact that 
ri.j > and r k ,j> > imply that 



j' < i, from (|75l l and Lemma [2] and [3] we have 

Aj(f)i, k (t> k ,j > Aj'4>i,j', and Ajifajfaj' > Ajfaj, (76) 

which lead to 4>i lk (f) kl i > 1, contradicting Lemma @] thus this 
is an impossible case. Case (2) i < j' < k, from (l75l l and 
Lemma |2] we have 

Aj<t>i,j'<pj',k<frk,j > Aj'<p it j>, and Ajxj) k) j, > Aj<f> kt j, 

(77) 

which lead to 4>j* k 4> k j> > 1, thus this is another impossible 
case. Case (3) j' > k, from ( |75T l and Lemma [2] we have that 
for this case 



Aj^i ik 4> kt j > Aj/fakfaj' 



(78) 



thus the new solution that r' k j = r k j' and r' k = does not 



k,j' 



decrease the quantity being optimized. Thus the conditions 
(i), (ii) and (iii) are indeed satisfied simultaneously by some 
optimal solution. The lemma is proved. ■ 
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Ajfaj > Aju 



and Aj,tp k ji > Aj<f> k<: 



(75) 



The three cases are now discussed individually next. Case (1) 



